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The present invention provides a method and apparatus for linking real-time data witfi audiovisual content to enable a user to make 
selections, manipulate data, and to execute functions interactively through an audiovisual display unit based upon audiovisual content being 
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audiovisual programs are synchronized with associated overlay functions and interface data. A method is provided to make overlay functions 
available to the user through an audiovisual display during the actionable events that occur in a program being viewed by the user. A 
user interface for each overlay function that may be performed, which user interface may or may not be visible to the user, is overlaid on 
and synchronized with die program being shown. Interaction with selections may be made using any of several known mediods such as a 
screen menu or a cursor controlled by a remote control device. 
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INTERACTIVITY WITH AUDIOVISUAL PROGRAMMING 

Background of the Invention 

1 . Fieid of the Invention: 

The present invention relates to the field of interactive audiovisual systems. More 
particularly, the present invention relates to a method and apparatus for linking real-time data 
with audiovisual content to enahle a user to make selections, manipulate data, and to execute 
functions by interacting with the audiovisual content being shown at the time. 

2. Art Background; 

With the development of computer technology, interactive systems are possible diat 
allow users to retrieve data and transact business through various means. Through the 
telephone, without speaking to a live operator, users can retrieve various types of 
information, order pay-per-view cable programs, activate credit cards, obtain customer 
support for products, order movie tickets and listen to and buy musical selections. Using the 
Internet and commercial online services, users are able to read news reports, view 
photographs and video clips, listen to sound files, send electronic mail, converse and 
videoconference in real time with users in remote locations, view sports statistics, buy 
products, generate maps, and retrieve archived data. 

Often a user may decide to use the telephone or an online service to perform such 
tiinaions based upon programming viewed ot} television. For example, a user may view a 
music v'ldGO on television and then decide to use an automated telephone service or the 
Internet to order an album by the artist shown. While viewing a tbotball game, a user might 
be prompted to use the Internet to retrieve statistics for a certain player or team. Users 
desire to have the ability to pertbrm these ftinctions and new functions using the television 
instead of having to use a different device such as a computer or telephone. 
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The advances in digitization and digital compression of data, improvements in 
transmission media such as fiber optics, and the combination of computer technology with 
television and audiovisual systems has fostered the development of interactive audiovisual 
systems. In interactive audiovisual systems, broad bandwidth permits two-way 
communication, or "interactivity," between the user and a source provider. Such systems 
allow the user, utilizing a remote control device in conjunction with a set-top box coupled to 
a cable television, satellite direct broadcast or other communication system, to select a 
desired movie or program to be viewed, request services and perform other ftinctions such as 
those that may be pertbrmed today using telephones and computer networks. Such systems 
generally require the user to access such ftjnctions by using channels separate from channels 
on which programming is shown and/or by using menus to access such functions. Often a 
cursor system is used to allow the user to perform functions using a remote control. 

While such systems enable selection of programs and other functions through the 
television, the selection is not enabled as a hinction of the content appearing on the television 
screen. The user must enter a certain mode or otherwise enable a ftinction which is always 
available to the user upon request. The present invention provides a method for enabling 
functions only at specified times based upon the content of audiovisual programming being 
shown to the user at the time. Furthermore, the functions enabled can be made to depend 
upon the content of the present audiovisual programming, so that the functions can be tailored 
to specific moments or events in an audiovisual presentation. The present invention presents 
the user with the ability to interact with the programming by selecting items appearing in the 
programming shown on the screen with a cursor, by making selections when prompted 
verbally by an on-screen host, or by other means. It will be understood by those skilled in 
the art that the present invention may be used for a variety of purposes, some of which are 
described in detail herein. 

Summary of the Invention 

Hie present invention provides a method and apparatus for enabling a user of an 
interactive audiovisual system to perform functions in real time based upon the a^ntent of 
audiovisual programming being shown to the user at the time ("overlay functions"). 
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The present invention provides a method and apparatus tor linking real-time data widi 
audiovisual content to enable a user to make selections, manipulate data, and to execute 
functions interactively through an audiovisual display unit based upon audiovisual content 
being displayed at the current time. Put another way, audiovisual content is presented in 
synchronization with selected overlay functions so that the user has the ability to interact with 
the audiovisual content by performing said overlay functions associated with selected events 
that occur in the programming ("actionable events"). 

A method and apparatus are provided to make overlay- functions available to the user 
through the audiovisual display during the actionable events that occur in programming being 
viewed by the user. A user interface for each overlay function that may be performed, which 
user interface may or may not be visible to the user, is overlaid on and synchronized with the 
programming being shown. Interaction with selections may be made using any of several 
known methods such as a screen menu or a cursor controlled by a remote control device. 
For example, while viewing a movie starring a famous celebrity, the set of overlay functions 
may include the ability to retrieve in an overlay window a list of recent movies starring the 
celebrity by selecting the celebrity's name during the opening credits of the movie, or 
whenever the celebrity appears on the audiovisual display. In some instances, a visible cue 
may exist to indicate that a function may be performed, such as a visible graphical outline 
appearing on the audiovisual display around an item that may be selected. As another 
example, during a football program, a user may be able to retrieve a team roster by selecting 
the name of a team when shown, or the career statistics of a player by selecting that player. 
As another example, while viewing a television program which reviews current movies, a 
user would have the ability to access an Internet World Wide Web site for a given movie by 
making a selection while that movie is being reviewed. As another example, the present 
invention may be used to implement an audiovisual program guide that allows a user to select 
programming being shown at the time by giving the user the ability to select one of several 
areas on the user's television screen, with each of the several areas displaying a different 
program that is available to the user at the time. 



In the present invention, a set of overlay ftinctions ("overlay function set") is 
associated with each program to be shown to users. The overlay functions aiuld include a 
variety of retrievable information and interactive ftinctions that will be available to the user 
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while the user is viewing the program. Each program may have unique overlay ftmction sets 
since the overlay ftinctions made available are based upon the unique content of each 
program.. For each actionable event in a program, the operator will determine the overlay 
function set that will be made available to the user, and the characteristics of the interface 
that will both present the availability of the overlay fiinctions to the user and allow the user to 
access the overlay functions. Overlay ftmction sets and interface data for a plurality of 
programs are stored in memory. When a program having overlay ftmction sets and interface 
data associated with it is sent by the source provider to a user (whether requested on demand, 
shown at a pre-<letermined time, or otherwise), the overlay ftmction sets and interface data 
for the program, which may also be sent by the source provider, are synchronized with the 
program and loaded into the set-top box. The processor in the set-top box reads the overlay 
function set and interface data and then both implements the interface and enables the overlay 
functions in real time and in synchronization with the associated actionable event in the 
programming. As the user interacts with the program through the synchronized interface, the 
set-top box implements the relevant overlay functions selected by the user. Alternatively, the 
set-top box may communicate with the source provider as necessary to implement certain 
overlay functions selected by the user. 
Brief Description of the Drawin i[ ?s 

Figure I is a functional block diagram of a head end of an interactive television 
system utilizing the teachings of the present invention. 

Figure 2 is a functional block diagram of a remote user location of an interactive 
audiovisual system utilizing the teachings of the present invention. 

Figure 3 shows one possible implementation of synchronization data. 

Figure 4 is a flowchart of the detailed process performed by the processing unit in the 
head end of an interactive audiovisual system in one embodiment of the present invention. 

Figure 5 is a flowchart of the detailed process performed by the processing unit in the 
set-top box in one embodiment of the present invention. 
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Figure 6 illustrates the basic protocol fields that facilitate the synchronization of 
overlay function sets and interface data with actionable events. 
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Detailed Description of the Invention 

I . General Considerations 

In the present case, the operations described herein are machine operations pertbrmed 
automatically or in conjunction with a human operator. Useftil machines tor performing the 
operations of the present inventions include general purpose digital computers, digitally 
controlled displays, or other similar devices such as set-top box control systems, video on 
demand servers and the like. The present invention relates to method steps tor operating an 
interactive audiovisual system and pn»cessing electrical or other physical signals to generate 
other desired physical signals. 

The present invention also relates to apparatus for performing these operations. This 
apparatus may be specially a)nstructed for the required purposes or it may comprise general 
purpose computers selectively activated or reconfigured by computer programs stored in the 
computers. The method steps presented herein are not inherently related to any particular 
computer or other apparatus. In particular, various general purpose machines may be used 
with programs in acairdance with the teachings herein, or it may prove more convenient to 
construct more specialized apparatus to perform the required steps. 

Additionally, no particular programming language has been indicateti for carrying out 
the various methods described herein. Each designer of a particular computer, interactive 
multimedia system, or other display apparatus will be aware of a language which is most 
suitable for his or her immediate purposes. Because the computers, servers, processing units, 
set-top boxes and other devices which may be used in practicing the present invention 
consists of many diverse elements, no detailed program listing has been provided. It is 
considered that the operations and other procedures described herein and illustrated in the 
accompanying drawings are sufficiently disclosed to enable one of ordinary skill to practice 
the present invention. 

In a typical interactive audiovisual system, a head-end is coupled to a wire or 
wireless, analog or digital, communication medium such as cable, satellite, microwave, 
terrestrial wave or optical fiber, which is in turn coupled to a plurality of users. At the 
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location of each user, a set-top box is coupled to the communication medium for receiving 
programming and data from the head end. In some systems it may be necessary for the set- 
top box to send data back to the head end via the same communication medium or a different 
communication medium. For example, a satellite signal could be used to transmit 
programming and data from the head end to the set-top box, while a wire telephony system 
may be used to send data from the set-top box back to the head end. The set-top box is also 
coupled to a television or other audiovisual display unit to enable the presentation of 
programming and data sent from the head end. The set-top box may include a processing 
unit coupled to a memory and other electronic modules such as decoders, decompressors and 
audiovisual generators. Computer programming and data files are stored within the memory 
and accessible by the processing unit. The processing unit generates and displays a graphical 
user interface on the user's audiovisual display unit, and controls any interactive 
communication between the user and the head end. Typically the user's system will include a 
user input device, such as a remote control device, for use in conjunction with the set-top box 
to enable the user to make selections, change channels and settings, and perform other 
ftinctions. 

2. General System Configuration 

One embodiment of the present invention may be described with reference to Figures 
1 and 2. Figure 1 is a functional block diagram of a head end of an interactive television 
system utilizing the teachings of the present invention. Figure 2 is a functional block 
diagram of a remote user location of an interactive audiovisual system utilizing the teachings 
of the present invention. In operation, as one of the first tasks in implementing this 
embodiment of the present invention, the entire length of each program with which users will 
be able to interact is indexed with sequential a)de data, which sequential code data may 
correspond to the elapsed time of each said program during its presentation. For example, 
the SMPTE time aide and the Sony RC time a)de are well-known in the art as forms of 
sequential aide data. Once the program has been sequentially a)ded, any actionable event in 
a program can be uniquely identified by the beginning and ending sequential codes during 
which the event occurs. It is to be noted that data other than time may be used to index the 
program sequentially, such as, for example, the number of digital bytes read, the number of 
film frames elapsed, or some other non-time-based means for a)unting. 
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As a next step in implementing the system illustrated in Fig. I, the program may be 
viewed by a human operator, before the program is to be transmitted to users, so that the 
operator may determine and rea)rd the sequential codes for the actionable events. For 
example, in a certain program it may be determined that "silver spaceship appears at 
sequential codes 1:13:43 through 1:14:17" or, in a hosted program guide, that "the on-screen 
host discusses Westerns at sequential codes 2:42:02 through 2:48:25." At this point there 
exists a set of data containing unique indicators tor each actionable event in the programming 
in the form of sequential a)de data associated with each of those actionable events. 

After the actionable events in a given program have been identified using the 
associated sequential code data, the overlay functions diat will be made available to the user 
and the interface through which the availability of the overlay functions will be presented to 
the user and through which the user will access the overlay trmctions, can be created for the 
program. Instructions to be used by the set-top box to allow the user to select overlay 
functions will be included as part of the interface data. As one example of the 
implementation of interface data, user input can be determined by defining a screen area of 
the user's audiovisual display unit which may be actively selected by a user, and 
implementing the corresponding overlay function if the user selects the defined screen area 
using a cursor or some other means for indicating the selection of a certain screen area. For 
example, the two-dimensional (X, Y) screen coordinates defining a selectable area of the 
screen may be sent to the set-top box as part of the interface data. Alternatively, the 
interface data defining a selectable screen area might include the two-dimensional coordinates 
of a single point on the screen and the radius of a circle centered at the single point. The set- 
top box may be programmed to generate the areas defined by the interface data. In addition 
to defining the selectable screen areas, the interface data may also include visual indicators, 
or overlays, which will appear over the program shown to the viewer during the actionable 
event to indicate where the selectable screen areas are located. For example, a certain object 
appearing on the screen may be highlighted in a certain transparent color to indicate to the 
user that the object may be selected with a cursor to perform an overlay ftinction while the 
object is highlighted. As another example, the interface data may include a graphical, 
animated or live-action ia)n that will appear at a certain area on the screen while the 
actionable event is occurring. In this example, the location of the ia)n would coincide with a 
selectable screen area so that the user may implement an associated overlay function by 
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selecting the icon. As another example, the interface data may be used to implement an 
audiovisual program guide that allows a user to select programming being shown at the time. 
The user's screen may separated into two or more screen areas, with each of the areas 
displaying a different program, or an icon or other representation of a different program, that 
is available to the user at the time. As described above, the interface data may include the 
two-dimensional (X,Y) coordinates of each screen area, as well as a visual indicator, or 
overlay, which will appear over the selectable screen area coinciding with the position of the 
user's cursor at the time. When the user moves the cursor to another selectable screen area, 
the visual indicator would appear over the new selectable screen area, which may then be 
selected by the user with the cursor to access the programming represented in that selectable 
screen area. 

As another example of the implementation of interface data, the user might perform a 
function by pressing a special button on a remote control device when a text message 
generated by the set-top box appears at the bottom of the screen. In such a case, the 
interface data could include the text message to be displayed, or the interface data may 
include instructions to the set-top box to generate the text message. 

It is to be understood that the interface need not include a graphical overlay to 
indicate an actionable event. Users may be prompted by other means to perform overlay 
functions during actionable events. For example, a program may include a speaking host 
(who may or may not appear on screen) that tells the user which overlay functions may be 
performed at the time as well as how the user may perform the ftinctions. Alternatively, the 
interface data may include audio data which is to be heard by the user when overlay functions 
are enabled. 

Aflter the operator has created the interface data and overlay hinction set that will be 
made available to the user for a given actionable event, it will be necessary to uniquely 
associate the interface data and overlay ftinction set with the related actionable event. The 
interface data and overlay function set for a given actionable event may be uniquely 
associated with the actionable event by using the same sequential code data to identify the 
actionable event, the related overlay function set and the related interface data. This may be 
accomplished by assigning the sequential a)de data for the actionable event to the related 
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interface data and overlay ftinction set. By assigning sequential code data to the interface 
data and overlay function set that matches the sequential code data of the related actionable 
event, the interface data and overlay function set can be retrieved from memory at the proper 
time and synchronized with the related actionable event by comparing the sequential code 
data assigned to the interface data and overlay function set with the current sequential aide 
data during the presentation of the program to the user. Accordingly, it is understood that 
for each sequentially coded program, synchronization data is created by a human operator 
which data contains the sequential code data for the actionable events in the programming and 
the location in memory of interface data and overlay ftinction set to be retrieved and 
presented to the user when the sequential code data for each actionable event is encountered. 
Those skilled in the art will understand that the synchronization data may take the form of a 
look-up table. 

Figure 3 shows one way in which synchronization data to be accessed by the 
processing unit (12) may be structured. In Figure 3, the synchronization data is depicted as a 
table of addresses of the calls for the overlay ftinction set and interface data associated with 
each actionable event in a program. It should be understood that additional attributes may be 
included in the synchronization data shown in Figure 3, such as a subset of user set-top box 
addresses for which a given actionable event will or will not be actionable. For example, if a 
certain group of users are not authorized to select overlay functions for a given event, the 
synchronization data may contain the set-top box addresses tor those unauthorized users so 
that the processing unit can be instructed not to send the overlay function set and interface 
data for the actionable event to the unauthorized users. 

Referring now to head end (2) shown in Figure 1, the first embodiment of the present 
invention uses sequentially coded programming received at the head end (2) by various means 
including transmission via satellite (4) from a remote location, or such sequentially a)ded 
programming may be stored in signal storage (6) at the head end. It is to be understood diat 
sequentially coded programming may be stored in a variety of formats, including but not 
limited to MPEG ft)rmat, DVD, videtuape, or other formats. A switcher (8) is provided 
which switches between the programming sources depending upon which is desired to be 
transmitted to the user. It is understood by those skilled in the art that the switcher may take 
many forms. The switcher transmits the sequentially coded program to a signal combiner 
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(10). It is understoc^d that the signal a)mbiner is well-known in the art and may be a 
multiplexer, a line inserter or other component. 

The sequentially coded program is also transmitted to a processing unit (12), which 
may include components such as random access memory, processors, co-processors, and 
interface drivers. Figure 4 illustrates the detailed process performed by the processing unit 
(12) in die present embodiment. At block (60), the processing unit (12) retrieves the 
synchronization data for the program from memory (14). The synchronization data is 
predetermined and may be stored in memory (14) by a human operator or downloaded to 
memory automatically from the source of the sequentially coded program with which the 
synchronization data is associated, prior to the transmission of the program. At block (62), 
the processing unit (12) reads the sequential code data associated with the sequentially aided 
program. At block (64), the processing unit (12) references the sequential code data to the 
synchronization data. At block (66), the processing unit (12) determines whether an overlay 
function set and interface data exist for the current sequential aide data (Le^, whether there is 
an actionable event occurring in the programming). At block (68), if an overlay fxinction set 
and interface data exist for the sequential a^de data read by the processing unit (12), the 
processing unit (12) retrieves the interface data from memory (16) and die overlay function 
set from memory (18). If an overlay function set and interface data do not exist for the 
sequential code data being read, the processing unit prepares itself to read the next sequential 
code data. The overlay function set and interface data is predetermined and may be stored in 
memories (16 and 18) by a human operator or downloaded to memory automatically from the 
source of the sequentially a>ded program with which the synchronization data is associated, 
prior to the transmissii>n of the program. At block (70), after retrieving the interface data 
and the overlay function set from memory, the processing unit (12) sends the overlay 
ftinction set and interface data to the signal combiner (10), which combines the overlay 
function set and interface data with the sequentially coded program so that the combined 
signal output by the signal aimbiner (10) a)ntains the interface data and the overlay ftinction 
set synchronized with the acti{)nable event identified by the sequential a)de data read by the 
processing unit (12). The signal aimbiner (10) sends the combined signal to the head end 
transmission equipment (22) for transmission to one or more remote user locations (as shown 
in Figure 2) via communication medium (30). It is understood that head end transmission 
equipment may take various forms and is well-known in die art. It should ftirther be 

11 

SUBSTITUTE SHEET (RULE 26) 



wo 98/28916 



PCTAJS97/23751 



understood that, in this embodiment, the synchronization of the interface data and the overlay 
function set with the associated actionable event occurs in the head end (2). Alternatively, 
such synchronization may occur at a remote location from which the synchronized interface 
data, overlay function set and program may be transmitted to the head end (2) or a plurality 
5 of head ends for ultimate transmission to users as further described herein. 

It is to be understood that a number of unique protocols may be designed to 
implement the synchronization of overlay function sets and interface data with actionable 
events as disclosed in the present invention. Figures 6, 6a, 6b and 6c illustrate basic protocol 
fields, to be employed in a transport layer, that may be used to facilitate the synchronization 

10 process. The signal combiner structures the a)mbined signal as shown in Figure 6. The 

sequential code data for a given actionable event may be sent first, followed by die overlay 
function set for the actionable event, followed by the interface data for the actionable event. 
As shown in Figure 6a, the sequential code data tor the actionable event may consist of the 
start and end codes for the actionable event. As shown in Figure 6b, the overlay function set 

15 may consist of the calls to be made by processing units to implement each available overlay 

function. As shown in Figure 6c, the interface data may consists of screen indicator 
attributes such as polygon shape, location, format, style and transitions. 

The apparatus of the remote user location is shown in detail in Figure 2. The 
combined signal containing the program, overlay function set and interface data is received at 

20 a receiving unit, which may be a set-top box (34) as shown in Figure 2, by a network 

interface (36) which sends the a)mbined signal to a decoder (38) which separates the program 
from the overlay function set and interface data sent from the head end (2), The decoder 
sends the program to a signal combiner (50) which sends the program to the audiovisual 
display unit (52) t()r presentation to the user. Simultaneously, the dea>der sends the 

25 synchronized interface data and overlay function set to a processing unit (40). Figure 5 

illustrates the detailed process performed by the processing unit (40) in the present 
embodiment. At block (72), the processing unit (40) receives the interface data and overlay 
function set. At block (74), the processing unit (40) reads and formats the interface data. At 
block (76), the processing unit (40) implements the user interface in accordance with the 

30 instructions contained within the interface data, including sending visual, audible, textual, and 

other material to the signal combiner (50) tor presentation to the user in synchronization with 
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the actionable event being sent to the audiovisual display unit (52). At bhjck (78), the 
processing unit (40) simultaneously reads the overlay function set and enables overlay 
ftinctions in synchronization with the actionable event being sent to the audiovisual display 
unit (52) so that the user is able to implement the various overlay ftinctions. The user may 
implement those ftinctions by making selections using a remote control device (48) which 
transmits input signals to a control interface (46) ampled to the processing unit (40). At 
block (80), the processing unit (40) determines whether the user has selected an overlay 
ftinction. At block (82), if the user has selected an overlay ftinction, the processing unit (40) 
causes the selected overlay ftinction to be performed. One or more of the overlay ftinctions 
selected by the user may be performed in the set-top box (34) by the processing unit (40), 
which may include components such as interface drivers, processors, and co-processors and 
may include random access memory (42) capable of running applications that may be 
accessible as part of the overlay functions available to the user. Such applications may be 
pre-loaded in the set-top box or downloaded from the head end as needed. Alternatively, one 
or more of the overlay functions selected by the user may be performed at the head end (2) 
or other remote location. Signals received from the user at the control interface (46) by the 
processing unit (40) may be sent through the network interface (36) via communication 
medium (32) to a transaction processor (24) located at the head end (2), which causes the 
implementation of the overlay functions. It should be understood at this point that . 
communication medium (32) and communication medium (30) may be the same medium or 
they me be different. For example, communication medium (30) may be a satellite signal, 
while communicatit)n medium (32) may he electrical wire, or alternatively, a two-way optical 
fiber may be used as the sole c{)mmunication medium between the head end (2) and a 
plurality of remote user locaticms. The transaction processor may be coupled to Internet 
servers (28), databases (26), computer networks and other components to facilitate the 
implementation of the various overlay ftinctions selected by users. For example, if an 
overlay ftinction allows the user to connect to a world-wide web site containing information 
related to an actionable event in the programming being presented, the transaction processor 
may be connected to an Internet server to allow the user to access the world-wide web site. 
As another example, if an overlay ftinction allows the user to request to be reminded when a 
ftiture program is about to begin, the transaction processor may be connected to a computer 
system set up to store unique user addresses and program reminder information to be sent to 
such addresses. At block (84). if the user did not select an overlay ftinction, the processing 
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unit (40) determines whether the current actionable event has ended. If the actionable event 
has not ended, the processing unit (40) continues to determine whether the user has selected 
an overlay function. If the actionable event has ended, the processing unit (40) resets and 
awaits the next actionable event. 

In a sea^nd embodiment of the present invention, the synchronization of the interface 
data and the overlay ftinction set with the associated actionable event occurs in the set-top box 
(34) instead of in the head end (2). In such an embodiment, referring again to Figure K the 
processing unit (12) may determine from data received prior to receiving the sequentially 
coded program whether overlay function sets and interface data are available tor the program 
(1^, whether there are one or more actionable events occurring in the programming). If 
overlay function sets and interface data are available, the processing unit (12) retrieves them 
from memory (16 and 18) and sends them to the set-top box (34) at some time prior to the 
transmission of the program to the set-top box (34). The processing unit (12) also retrieves 
synchronization data from memory (14) which, as further described below, will allow the set- 
top box (34) to synchronize overlay function sets and interface data with actionable events 
based upon sequential code data. It is to be understood that the synchronization data, overlay 
function sets and interface data may be stored with the associated pn)gram (in signal storage 
(6) or in storage at the location from which satellite transmitted programs are transmitted) and 
transmitted immediately prior to the transmission of the program to the switcher (8) and 
subsequently to the processing unit (12) and on to the set-top box (34). 

The synchronization data, overlay function set and interface data, and the sequentially 
coded program, are received at the set-top box (34) by network interface (36) coupled to 
decoder (38). The decoder (38) sends the synchronization data, interface data and overlay 
function data to a processing unit (40) which stores them in memory (44). The processing 
unit (40) performs the same process detailed in Figure 4 as described with respect to the first 
embodiment, whereby the processing unit (40) is programmed to read the sequential code 
data associated with the program and reference it to the synchronization data stored in 
memory (44) to determine whether overlay function sets and interface data exist tor die 
sequential code data read by the processing unit (40) (i.e. . whether there are actionable events 
occurring in the programming). If overlay t\jnction sets and interface data exist tor the 
sequential code data read by the processing unit (40), the processing unit retrieves the overlay 
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ftinction sets and interface data from memory (44). The processing unit (40) reads the 
interface data and implements the user interface in accordance with the instructions contained 
within the interface data, including sending visual, audible, textual, and other material to the 
signal combiner (50) for presentation to the user in synchronization with the actionable event 
5 being sent to the audiovisual display unit (52), The processing unit (40) also reads the 

overlay function set and enables overlay ftinctions in synchronization with the actionable 
event being sent to the audiovisual display unit (52) so that the user is able to implement the 
various overlay liinctions. 

10 In a third embodiment of the present invention, it should be appreciated that a 

program and related interface data and overlay function sets may be stored in transportable 
media such that actionable events in the programming are synchronized with associated 
interface data and overlay function sets prior to presentation to a user. In such an 
embodiment, the synchronization process described in the first embodiment and shown in 

15 Figure I as occurring in the head end (2) may be performed as described, except that the 

combined signal which is output by the signal combiner (10) may be stored in storage (20) 
for later transmission from the head end (2) as already described tor the first embodiment, or 
for transfer to copies of transportable media, such as videotape, compact disc, DVD or other 
formats. Alternatively, the combined signal which is output by the signal combiner (10) may 

20 be transmitted to the remote user location, as described in the first embodiment, and then 

stored in transportable media by an appropriate recording/playback device (54) coupled to 
decoder (38) through external device interface (56). As shown in Figure 2, during playback 
from transportable media by rea)rding/playback device (54), the combined signal is 
transmitted through external device interface (56) to decoder (38) and is processed as 

25 described in the first embodiment presented herein. Aldiough the transmission of the 

program would originate in the user's home or other remote location from the 
recording/playback device (54), it would still be possible for one or more of the overlay 
ftmctions selected by the user to be performed at the head end (2) or other remote location 
instead of in the set-top box (34). Signals received from the user at the a>ntrol interface (46) 

30 by the processing unit may be sent through the network interface (36) via communication 

medium (32) to a transaction process(»r (24) located at the head end (2) or other remote 
location, which would implement the overlay ftinctions. 
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In yet another embodiment of the present invention, actionable events tor a program 
may be selected, and interface data and overlay txinction sets may be created or selected tor 
those actionable events and made available to users, during the transmission of the program. 
In this embodiment, a human operator using a general purpose computer located at a head 
end location (2) may select overlay functions and create an interface, or select a pre-existing 
interface, for a given event that the operator wishes to make actionable for users. After 
selecting the overlay functions and the interface for the current event which the operator 
wishes to make actionable, the operator instructs die processing unit (12) in the head end to 
immediately send the overlay t\inction set and interface data to the signal combiner (10) to be 
synchronized with the event being transmitted to users at the time. In this embodiment, it is 
not necessary for the program to be sequentially a)ded in advance of transmission since the 
operator determines when the overlay function sets and interface data should be sent to users. 
Upon receipt at the set-top box (34), the program, the interface data and the overlay tiinction 
sets are handled in the same fashion as the first embodiment described herein, whereby the 
overlay function sets and interface data are processed by the processing unit (40) and 
presented to users immediately. 

There has been described novel methods and apparatus for linking real-time data w^th 
audiovisual content to enable a user to make selections, manipulate data, and to execute 
functions by interacting with the audiovisual content being shown at the time. It is evident 
that those skilled in the art may now make numerous uses and modifications of and 
departures from the specific embodiments described herein without departing fr(»m the 
inventive concepts. Consequently, the invention is to be construed as embracing each and 
every novel feature and novel combination of features present in or possessed by the 
apparatus and methods herein disclosed and limited solely by the spirit and scope of the 
appended claims. 
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We claim: 

1. A method of implementing an interactive audiovisual system wherein one or 
more users has the ability to interact with said interactive audiovisual system by making 
selections while viewing linear programming, said linear programming including one or more 
actionable events, the method comprising the following steps: 

5 selecting (me or more events of said linear programming to be 

actionable events, 

associating, with each of said selected one or more actionable events, 
one or more overlay functions to be accessible to one or more of said users 
during such actionable event, 

^0 associating, with each of said selected one or more actionable events, 

interface data for enabling one or more of said users to perform said one or 
more overlay functions, 

providing said linear programming, including said selected one or 
more actionable events, said associated interface data and said associated one 
'5 or more overlay functions to one or more of said users such that each of said 

selected one or more actionable events is synchronized with its associated 
interface data and its associated one or more overlay ftinctions. 

2. The method of Claim 1 wherein said step of providing said linear 
programming further includes providing said linear programming via a wire network. 

3. The method of Claim I wherein said step of providing said linear 
programming further includes the step of providing said linear programming via a wireless 
network. 
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4. The method of Claim I ftinher including the step of providing said linear 
programming in an analog format. 

5. The mediod of Claim 1 further including the step of providing said linear 
programming in a digital format. 

6. The method of Claim I wherein said step of providing said linear 
programming ftirther includes the step of providing said linear programming via analog 
terrestrial signal. 

7. The method of Claim 1 wherein said step of providing said linear 
programming further includes the step of providing said linear programming via digital 
terrestrial signal. 

8. The methtxJ of Claim 2 wherein said step of providing said linear 
programming via a wire network further includes providing said linear programming via a 
cable network. 

9. The method of Claim 1 wherein said step of providing said linear 
programming further includes the step of providing said linear programming via analog 
satellite signal. 

10. The method of Claim 1 wherein said step of providing said linear 
programming further includes the step of providing said linear programming via digital 
satellite signal. 

11. The method of Claim 1 wherein said step of providing said linear 
programming ftirther includes the step of providing said linear programming via analog 
microwave signal. 

12. The method of Claim I wherein said step of providing said linear 
programming ftirther includes the step of providing said linear programming via digital 
microwave signal. 
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13. The method of Claim 5 wherein said step of including said linear 
programming in a digital format includes the step of providing said linear programming in an 
MPEG format. 

14. The method of Claim I wherein each of said selected ()ne or more actionable 
events is synchronized with its associated interface and its associated one or more overlay 
ftmctions in a location remote from the locations of said users prior to providing said linear 
programming, including said selected one or more actionable events, said associated interface 
data and said associated one or more overlay functions to one or more of said users. 

15. The method of Claim 14 wherein said location remote from the locations of 
said users is a head end of an interactive audiovisual system. 

16. The method of Claim 1 wherein each of said selected one or more actionable 
events is synchronized with its associated interface and its associated one or more overlay 
ftmctions at the individual locations of each of said one or more users after the providing of 
said linear programming, including said selected one or more actionable events, said 
associated interface data and said associated one or more overlay ftjnctions from a remote 
location to the individual locations of each of said one or more users. 

17. The method of Claim 16 wherein said remote location is a head end of an 
interactive audiovisual system. 

18. The method of Claim I wherein each of said selected one or more actionable 
events is synchronized with its associated interface and its associated one or more overlay 
functions in a location remote from the locations of said users and stored in transportable 
media for later playback from said transportable media by appropriate recording/playback 
devices located at the individual location of each of said one or more users. 

19. The method of Claim I ftarther including the step of associating with said 
linear programming, sequential a^de data for indexing said linear programming during its 
presentation. 
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20. The method of Claim 19 wherein said sequential code data corresponds to the 
elapsed time of said linear programming during its presentation. 

21. The method of Claim 19 wherein said step of selecting one or more 
actionable events of said linear programming includes the step of using said sequential code 
data to identify said selected one or more actionable events. 

22. The method of Claim 19 ftirdier including the step of using said sequential 
code data to identify said selected one or more overlay functions. 

23. The method of Claim 19 further including the step of using said sequential 
code data to identify said interface data. 

24. The method of Claim 19 wherein said step of associating, with each of said 
selected one or more actionable events, one or more overlay ftinctions to be performed by 
one or more of said users during such actionable event, further includes the step of 
identifying each of said selected one or more actionable events and the said one or more 
overlay ftinctions to he associated therewith using the same sequential code data. 

25. The method of Claim 19 wherein said step of associating, with each of said 
selected one or more actionable events, interface data for enabling one or more of said users 
to perform said one or more overlay ftinctions, ftirther includes the step of identifying each of 
said selected one or more actionable events and said interface data to be associated therewith 
using the same sequential code data. 

26. The method of Claim 19 wherein said step of associating, with each of said 
selected one or more actionable events, one or more overlay functions to be performed by 
one or more of said u.sers during such actionable event, ftirther includes the step of creating 
synchronization data in which each set of said one or more overlay functions is identified 
using the same sequential code data used to identify the selected actionable event. 
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27. The method of Claim 26 wherein said synchronization data is in the form of a 
look-up table including the addresses of the calls for the overlay function set associated with 
each actionable event contained in said linear programming. 

28. The method of Claim 19 wherein said step of associating, with each of said 
selected one or more actionable events, interface data for enabling one or more of said users 
to perform said one or more overlay functions, further includes the step of creating 
synchronization data in which each such interface data is identified using the same sequential 
code data used to identify the selected actionable event. 

29. The method of Claim 28 wherein said synchronization data is in the form of a 
look-up table including the addresses of the interface data associated with each actionable 
event contained in said linear programming. 

30. The method of claim 19 wherein said step of providing said linear programming, 
including said selected one or more actionable events, said associated interface data and said 
associated one or more overlay functions to one or more of said users such that each of said 
selected one or more actionable events is synchronized with its associated interface data and 
its associated one or more overlay functions, further includes the steps of referencing said 
sequential code data to synchronization data for said linear programming to determine 
whether interface data and one or more overlay functions exist for the current sequential code 
data, and providing any such existing interface data and overlay functions for a>mbination 
with the actionable event tor the current sequential code data, such that each of said selected 
one or more actionable events is synchronized with its associated interface data and its 
associated one or more overlay functions. 

31. The method of Claim 30 wherein said synchronization data is in the form of a 
look-up table including the addresses of the calls tor the overlay fiinctiim set and interface 
data associated with each actionable event contained in said linear programming. 

32. The method of Claim I wherein said one or more overlay functions include 
the capability to retrieve data. 
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33. The method of Clann I wherein said one or more overlay ftinctions include 
the capability to connect to the Internet, on-line services, databases, and other real-time 
transactional processing systems. 

34. The method of Claim I wherein said one or more overlay functions include 
the capability to request the immediate receipt of programming, 

35. The method of Claim 1 wherein said one or more overlay functions include 
the capability to schedule the receipt of programming at a later time. 

36. The method of Claim 1 wherein said one or more overlay tiinctions include 
the capability to schedule the recording of programming at a later time. 

37. The method of Claim 1 wherein said one or more overlay functions include 
the capability to order products and/or services. 

38. The method of Claim 1 wherein said one or more overlay functions include 
the capability to input data. 

39. The method of Claim 1 wherein said interface data includes text, audible 
items, and/or static, animated or live-action visual items, 

40. The method of Claim i wherein said interface data includes screen 
a)ordinates for areas of a user's audiovisual display unit. 

41. A method of implementing an interactive audiovisual system wherein one or 
more users has the ability to interact with said interactive audiovisual system by making 
selections while viewing linear programming, said linear programming including one or more 
actionable events, the method ci)mprising the following steps: 

associating with said linear programming, sequential code data for 
indexing said linear programming during its presentation. 
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selecting one or more events of said linear programming to be 
actionable events, 

identifying said selected one or more actionable events using said 
sequential code data, 

associating, with each of said selected one or more actionable events, 
one or more overlay functions to be accessible to one or more of said users 
during such actionable event by identifying each of said selected one or more 
actionable events and the said one or more overlay functions to be associated 
therewith using the same sequential code data, 

associating, with each of said selected one or more actionable events, 
interface data for enabling one or more of said users to perform said one or 
more overlay functions by identifying each of said selected one or more 
actionable events and said interface data to be associated therewith using the 
same sequential a)de data, 

providing said linear programming, including said selected one or 
more actionable events, said associated interface data and said associated one 
or more overlay functions to one or more of said users. 

42. The method of Claim 41 wherein said sequential code data corresponds to the 
elapsed time of said linear programming during its presentation, 

43. An interactive audiovisual system for providing, and enabling one or more 
users to interact with linear programming, said linear programming including selected one or 
more actionable events each having an overlay ftinction set and interface data associated 
therewith, and said linear programming being provided such that each of said selected one or 
more actionable events is synchn)nized with its associated overlay function set and its 
associated interface data, the apparatus a)mprising the following: 

a signal source for providing said linear programming. 
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a processing unit, coupled to said signal source, which is programmed 
to provide said interface data and said overlay function sets associated widi 
10 said one or more actionahle events, 

memory means, a)upled to said processing unit, for storing said 
overlay ftinction sets and said Interface data, 

a signal comhiner, coupled to said signal source and said processing 
unit, for combining said linear programming and said interface data and 
15 overlay ftinction sets, 

transmitting means, coupled to said signal combiner, for transmitting 
a signal comprising said linear programming in combination with said 
interface data and said overlay ftinction sets to one or more users at remote 
locations, 

20 one or more receiving units for receiving said signal from said 

transmitting means, each said receiving unit uimprising, 

decoding means for separating said linear programming from 
said interface data and said overlay ftinction sets, 

a sea>nd processing unit, coupled to said decoding means, 
25 which is programmed to read and format said interface data and to 

enable said overlay ftinction sets associated with said one or more 
actionable events, 

memory means, coupled to said second processing unit, for 
storing said overlay ftinction sets and said interface data, 

30 a second signal aimbiner, coupled to said decoding means 

and said second processing unit, for aimbining said linear 
programming and said interface, 
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an audiovisual display, coupled to each said receiving unit, for 
receiving said linear programming in combination with said interface from 
said second signal combiner tor presentation to each said user, 

input means, coupled to each said receiving unit, allowing each said 
user to select said one or more overlay functions when presented to said user. 

44. The apparatus of Claim 43 further including a transaction processor coupled 
to each said receiving unit, for processing overlay functions selected by each said user. 

45. The apparatus of Claim 43 wherein said receiving unit is a set-top box. 

46. The apparatus of Claim 43 wherein said input means is a remote control 
device or a keyboard. 

47. A head end of an interactive audiovisual system tor providing, and enabling 
one or more users to interact with, linear programming, said linear programming including 
selected one or more actionable events each having an overlay function set and interface data 
associated therewith, and said linear programming being provided such that each of said 
selected one or more actionable events is synchronized with its associated overlay function set 
and its associated interface data, the apparatus a)mprising the following: 



a signal .s<iurce for providing said linear programming. 



a processing unit, coupled to said signal source, which is programmed 
to provide said interface data and said overlay function sets associated wth 
said one or more actionable events, 



memory means, coupled to said processing unit, tor storing said 
overlay function sets and said interface data. 
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a signal aimbiner, coupled to said signal source and said processing 
unit, for combining said linear programming and said interface data and 
overlay function sets, 

transmitting means, coupled to said signal combiner, for transmitting 
a signal comprising said linear programming in combination with said 
interface data and said overlay txinction sets to one or more users at remote 
locations. 



48. A receiving unit for receiving a signal comprising linear programming including 
selected one or more actionable events each having an overlay function set and interface data 
associated therewith, and said linear programming being provided such that each of said 
selected one or more actionable events is synchronized with its associated overlay function set 
and its associated interface data, each said receiving unit comprising, 

decoding means for separating said linear programming from said 
interface data and said overlay function sets, 

a processing unit, coupled to said decoding means, which is 
programmed to read and format said interface data and to enable said overlay 
function sets associated with said one or more actionable events. 



memory means, coupled to said processing unit, for storing said 
overlay ftinction sets and said interface data, 

a signal aimbiner, coupled to said decoding means and said second 
processing unit, for combining said linear programming and said interface 
data. 



input means for allowing each said user to select said one or more 
overlay functions when presented to said user, and 
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output means tor transmitting said combined linear 
programming and interface data to an audiovisual display for 
presentation to each said user. 



49. An interactive audiovisual system for providing, and enabling one or more 
users to interact with, linear programming, said linear programming including selected one or 
more actionable events, identified using sequential code data, each having an overlay function 
set and interface data associated therewith by identifying each of said selected one or more 
actionable events and said overlay functicm set and said interface data to be associated 
therewith using the same sequential code data, the apparatus comprising the following: 



a signal s{)urce tor providing said linear programming. 



a processing unit, coupled to said signal source, which is programmed 
to read said sequential code data and provide said interface data and said 
overlay ftinction sets associated with said one or more actionable events, 

memory means, coupled to said processing unit, tt>r storing said 
overlay ftinction sets and said interface data, 

a signal combiner, coupled to said signal source and said processing 
unit, for combining said linear programming and said interface data and 
overlay f\inction sets, 

transmitting means, coupled to said signal combiner, for transmitting 
a signal comprising said linear programming in combination with said 
interface data and said overlay function sets to one or more users at remote 
locations, 



one or more receiving units for receiving said signal from said 
transmitting means, each said receiving unit comprising. 
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decoding means for separating said linear programming from 
said interface data and said overlay function sets, 



a second processing unit, coupled to said decoding 
means, which is programmed to read and format said interface data 
and to enable said overlay function sets associated with said one or 
more actionable events. 



memory means, coupled to said second processing unit, for 
storing said overlay function sets and said interface data, 

a second signal combiner, coupled to said decoding means 
and said second processing unit, for combining said linear 
programming and said interface, 

an audiovisual display, coupled to each said receiving unit, for 
receiving said linear programming in combination with said interface from 
said second signal combiner for presentation to each said user, 

input means, a)upled to each said receiving unit, allowing each said 
user to select said one or more overlay functions when presented to said user. 

50. The apparatus of Claim 49 further including a transaction processor a)upled 
to each said receiving unit, for processing overlay functions selected by each said user. 

51. The apparatus of Claim 49 wherein said receiving unit is a set-top box. 

52. The apparatus of Claim 49 wherein said input means is a remote control 
device or a keyboard. 



53. An interactive audiovisual system for providing, and enabling one or more 
users to interact with, linear programming, said linear programming including selected one or 
more actionable events, identified using sequential code data, each having an overlay function 
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set and interface data associated therewith by identifying each of said selected one or more 
actionable events and said overlay ftinction set and said interface data to be associated 
therewith using the same sequential code data, the apparatus comprising the following: 

a signal source for providing said linear programming and said 
overlay ftinction sets and said interface data, 

a processing unit, coupled to said signal source, which is programmed 
to provide said interface data and said overlay function sets associated with 
said one or more actionable events, 

transmitting means for transmitting a signal comprising said linear 
programming, said interface data and said overlay function sets to one or 
more users at remote locations, 

one or more receiving units for receiving said signal fn)m said 
transmitting means, each said receiving unit comprising, 

decoding means for receiving said linear programming and 
said interface data and said overlay function sets, 

a second processing unit, coupled to said decoding 
means, which is programmed to read said sequential code data, to 
read and format said interface data and provide said interface based 
upon said sequential code data, and to enable said overlay ftinction 
sets associated with said one or more actionable events based upon 
said sequential code data, 

memi)ry means, coupled to said second processing unit, for 
storing said overlay ftinction sets and said interface data. 
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a sea)nd signal combiner, coupled to said deaxling means 
and said sea)nd processing unit, for combining said linear 
programming and said interface, 

an audiovisual display, aiupled to each said receiving unit, for 
receiving said linear programming in combination with said interface from 
said second signal a>mbiner for presentation to each said user, 

input means, coupled to each said receiving unit, allowing each said 
user to select said one or more overlay ftinctions when presented to said user. 

54. The apparatus of Claim 53 further including a transaction processor coupled 
to each said receiving unit, for processing overlay functions selected by each said user. 

55. The apparatus of Claim 53 wherein said receiving unit is a set-top box. 

56. The apparatus of Claim 53 wherein said input means is a remote control 
device or a keyboard. 



57. An apparatus for recording linear programming with which one or more users 
will be able to interact, said linear programming including selected one or more actionable 
events each having an overlay ftinction set and interface data associated therewith, and said 
linear programming being provided such that each of said selected one or more actionable 
events is synchronized with its associated overlay ftjnction set and its associated interface 
data, the apparatus comprising the following: 

a signal source tor providing said linear programming, 



a processing unit, a)upled to said signal source, which is programmed 
to provide said interface data and said overlay tiinction sets associated with 
said one or more actionable events, 
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memory means, ampled to said processing unit, tor storing said 
overlay function sets and said interface data, 

a signal combiner, coupled to said signal source and said processing 
unit, for combining said linear programming and said interface data and 
overlay ftinction sets, 

recording means, coupled to said signal combiner, ftir recording said 
linear programming in aimhination with said interface data and said overlay 
function sets. 

58. An apparatus for recording linear programming with which one or more users 
will be able to interact, said linear programming including selected one or more actionable 
events each having an overlay function set and interface data associated therewith, and said 
linear programming being provided such that each of said selected one or more actionable 
events is synchronized with its associated overlay function set and its associated interface 
data, the apparatus comprising the following: 

a signal source for providing said linear programming, 

a processing unit, coupled to said signal source, which is programmed 
to provide said interface data and said overlay function sets associated with 
said (»ne or more actionable events, 

memory means, coupled to said processing unit, for storing said 
overlay function sets and said interface data, 

a signal combiner, a)upled to said signal source and said processing 
unit, for aimbining said linear programming and said interface data and 
overlay function sets, 

transmitting means, coupled to said signal combiner, tor transmitting 
a signal comprising said linear programming in ambination with said 
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interface data and said overlay function sets to one or more users at remote 
locations, 

one or more receiving units for receiving said signal from said 
transmitting means, 

recording means, coupled to said one or more receiving units, tor 
recording said signal comprising said linear programming in aimbination with 
said interface data and said overlay function sets. 

59. An apparatus for enabling a user to play back from storage recorded linear 
programming with which one or more users will be able to interact, said linear programming 
including selected one or more actionable events each having an overlay function set and 
interface data associated therewith, and said linear programming being provided such that 
each of said selected one or more actionable events is synchronized with its associated 
overlay ftinction set and its associated interface data, the apparatus a)mprising the following: 

playback means for accessing said stored linear programming and 
associated overlay ftinction set and associated interface data, 

a receiving unit coupled to said playback means, each said receiving 
unit comprising, 

decoding means for separating said linear programming from 
said interface data and said overlay function sets, 

a processing unit, coupled to said decoding means, 
which is programmed to read and format said interface data and to 
enable said overlay function sets associated with said one or more 
actionable events, 

memory means, coupled to said sea)nd processing unit, for 
storing said overlay ftinction sets and said interface data. 



32 

SUBSTITUTE SHEET (RULE 26) 



wo 98/28916 



PCT/US97/23751 



a signal combiner, coupled to said deaiding means and said 
processing unit, for combining said linear programming and said 
interface, 

an audiovisual display, a)upled to said receiving unit, for receiving 
said linear programming in combination with said interface from said signal 
combiner for presentation to each said user, 

input means, coupled to said receiving unit, allowing said user to 
select said tine or more overlay functions when presented to said user. 

60. The apparatus of Claim 59 further including a transaction processor coupled 
to each said receiving unit, for processing overlay functions selected by each said user. 

61. The apparatus of Claim 59 wherein said receiving unit is a set-top box. 

62. The apparatus of Claim 59 wherein said input means is a remote control 
device or a keyboard. 



33 

SUBSTITUTE SHEET (RULE 26) 



wo 98/28916 



PCT/US97/23751 




X 





1/5 


1 

• 1 
I 






1 

r-l.. 










mUNICATR 
MEDIUM 




11 








I ' 



§1 

1^ 



8 



I 1 



i*J ! 



Uj 

c3 



Co ! 



is 













ii 






















to 







SUBSnrUTE SHEET (RULE 26) 



wo 98/28916 



PCT/US97/23751 



2/5 



8 



CD 



i 




SUBSTITUTE SHEET (RULE 26) 



wo 98/28916 



PCT/US97/23751 





3/5 


FIG. 


5 


ACTIONABLE EVENT 
RANGE 


OVERLAY FUNCTION SET 
ADDRESSES 


INTERFACE DATA 

AUUnLooto 


00:08:32:05-00:12:15:32 


X00013256 
X00013368 
X00014262 
X00017350 




X00002426 
X00002854 


00:35: 12: 1 7-00:42:06:27 


X00018462 
X00018794 
X00019236 




X00003628 



60- 



FIG. 4 



62- 



64- 



RETRIEVE SYNCHRONIZATION DATA 



READ SEQUENTIAL CODE DATA 



REFERENCE SEQUENTIAL CODE 
DATA TO SYNCHRONIZATION DATA 



IS 

THERE AN 
OVERLAY FUNCTION 
SET AND INTERFACE DATA FOR" 
THE CURRENT SEQUENTIAL 
CODE DATA 

9 



68- 



70- 



RETRIEVE OVERLAY FUNCTION SET 
AND INTERFACE DATA FROM MEMORY 



SEND OVERLAY FUNCTION SET AND 
INTERFACE DATA TO SIGNAL COMBINER 



SUBSTITUTE SHEET (RULE 26) 



wo 98/28916 



PCT/US97/23751 



4/5 



FIG. 5 



BEGIN 



72- 



RECEIVE INTERFACE DATA AND 
OVERLAY FUNCTION SET 



74 



READ AND FORMAT 
INTERFACE DATA 



78 



L 



READ AND LOAD 
OVERLAY FUNCTION SET 



76- 



SEND INTERFACE DATA TO 
SIGNAL COMBINER TO BE 
COMBINED WITH PROGRAM 










SUBSTITUTE SHEET (RULE 26) 



wo 98/28916 



PCT/US97/23751 



FIG. 6 



5/5 

BASIC PROTOCOL 



SEQUENTIAL CODE DATA 



OVERLAY FUNCTION SET 



INTERFACE DATA 



FIG. 6A 



SEQUENTIAL CODE DATA 





EVENT START CODE 


EVENT END CODE 






\ MONTH:DAY:YEAR:HH:mSS:FF:TIME REFERENCE \ 



OVERLAY FUNCTION SET 



FIG. 6B 



CALLS FOR EACH OVERLAY FUNCTION IN THE SET 



INSTRUCTIONS FOR ACTIVATION OF EACH OVERLAY FUNCTION 



FIG. 6C 




IMPLEMENTATION OF INTERFACE DATA AT AUDIOVISUAL DISPLAY 



VISUAL AND AUDIO ATTRIBUTES: POLYGON SHAPE: LOCATION: 
FORMAT: STYLE: TRANSITIONS 



SUBSTITUTE SHEET (RULE 26) 



fXi'l'Li'TJ NT A TTOM A T OT7 A n/^TT n n^n^nT* 

iri 1 JLAiN A 1 iKJiH ALi o JliAKCH RKPQRT 


Inl YlonaJ Application No 

PCT/US 97/23751 


A. CLASSFICATION OF,SUBJECT MATtEr ~ 

IPC 6 H04N7/173 




According to IntemationaJ Patent ClassillcatlontlPC) or to both national dassification and IPC 




B. RELOS SEARCHED 


Minimum documentation searched (classification system foflowed by dassiflcation symbols) 

IPC 6 H04N 



Documentation searched other than minimum documentation to the extent that such documents are included In the fields searched 



Electronic data base consulted during the international search (name of data base and, where practical, search temts used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



us 5 517 257 A (DUNN MATTHEW M ET AL) 14 
May 1996 



see column 2, 
see column 4, 
see column 5, 
figure 3 



line 50 - column 3, line 23 
line 13 - line 46; figure 2 
line 47 - column 6, line 2; 



-/- 



1-17,40. 
41, 

45-49, 
51-53, 
55-59, 
61,62 



"xl Further docurtiBrts are listod In the oontlnualion of box C. 



ID 



Patent famiiy members are listed in annex. 



Special categories of cited documents : 

'A" document defining the general state ol the art which is not 
considered to be of particular relevance 

"E" earlier document but published on or after the international 
fib'ng date 

"L" document which may throw doiiJts on priority daim(s) or 
which is cited to establish the publication date of another 
citation or other special reason (as specified) 

•O" document referring to an oral disclosure, use, exhibition or 
other means 

T" document pubfished prior to the International fifing date but 



T later document published after the International filing date 
or priority date and not In conflict with the application but 
cited to understand the principle or theory underlying the 
invention 

■X" document ol particular relevance; the daimed Invention 
cannot be considered novel or cannot be considered to 
involve an inventive step when the document is ta)cen alone 

"Y" document ot particular relevance: the daimed invention 

cannot be cortsidered to involve an Inventive step when the 
document Is combined with one or more other such docu- 
ments, such combination being obvious to a person skiDod 
mUwarl 



Date of tiw actual compiellon of the International search 

17 April 1998 


Date of malUng of the international search report 

27/04/1998 


Name and mailing address of the ISA 

European Patent Office, P.B, 5818 Patentlaan 2 
NL-2280HVRijswqk 
Tel. (431-70) 340-2040. Tx. 31 651 epo nl. 
Fax: (+31-70) 340-3016 


Authorized officer 

Fuchs, P 



page 1 of 2 



INTERNATIONAL SEARCH REPORT 



Int Uonal Application No 

PCT/US 97/23751 



C^Contlnuatlon) DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document, with indication. wtiere appropriate, ol the relevant passages 



Relevant to daim No. 



EP 0 746 153 A (IBM) 4 December 1996 



see abstract 

see page 2, line 30 - page 4, line 29; 
figures 1,3 

see page 26, line 39 - page 27, line 8; 
claims 1,2; figures 13-15 

EP 0 711 076 A (AT & T CORP) 8 May 1996 



see the whole document 



1-5.13. 

40,41. 

43. 

47-49 , 
53,57-59 



1-17.37, 

40,41. 

43, 

47-49, 
53,57-59 



Fonn PCT/ISAS10 {conthuaOon of socond sheet) (Ji^ 1992) 



page 2 of 2 



INTERNATIONAL SEARCH REPORT 

Information on patent family memtiers 



tnt \(onal Applicatton No 

PCT/US 97/23751 



Patent document 
cited in search report 



Pubiication 
date 



Patent famBy 
nieiiil>er(s) 



Publication 
date 



US 5517257 A 



EP 0746153 A 



EP 0711076 A 



14-05-96 
04-12-96 



US 5648824 A 



US 5602597 A 
CA 2177918 A 
JP 8328800 A 



08-05-96 



CA 
JP 



2158293 A 
8214273 A 



15-07-97 



11-02-97 
01-12-96 
13-12-96 



04-05-96 
20-08-96 



Foim PCT/TSA^O (patsm famfly annex) (J^ 1S92) 



